In recent years, DVDs in which video and audio are digital-compressed and recorded have become widespread, and are widely used for easily playing various kinds of software such as movie and karaoke. A VTR which has been conventionally utilized played back such as a movie in the recorded order at the most as a function of playing a video. For a DVD, however, a function of playing a desired scene in a moment and the like can be provided. Further, while a program being broadcast can be simply viewed on TV in a prior art, the emergence of digital broadcasting enables viewers to answer quizzes at home by using a remote controller for a television when they are viewing a quiz show, for example. Moreover, when a drama is being viewed, profiles of actors on television can be displayed by merely pressing a button.
For such as the DVD and digital broadcasting, the MPEG (Moving Picture Experts Group) is used as an information compression technique for efficiently storing and transmitting video and audio. In MPEG, a construction of a signal used for multiplexing a plurality of digital signals which are constituents of digital contents such as video and audio into a bitstream is defined as a standard.
In this standard, a PCR (Program Clock Reference) or an SCR (System Clock Reference) which is time information for setting a value of an STC (System Time Clock) as a reference time on the decoder side in the MPEG system, to a value as intended on the encoder side, a PTS (Presentation Time Stamp) which is time information added for each unit of coding video or audio as well as time management information for playback and output, and a DTS (Decoding Time Stamp) which is time management information for decoding are coded into a bitstream, thereby performing synchronous playback with higher precision.
Here, a format of a video/audio stream for MPEG2 will be described with reference to FIG. 15. FIG. 15 is a diagram schematically illustrating a construction of the video/audio stream. In FIG. 15, a video/audio stream is constructed as a set of a video pack (V_PCK) and an audio pack (A_PCK). The V_PCK comprises a pack header and a video packet, and an SCR and the like are recorded in the pack header. The video packet further comprises a packet header and a video elementary stream. A stream ID, a DTS, a PTS and the like are recorded in the packet header and the video elementary stream comprises an I picture, a P picture, and a B picture which are bitstreams obtained by compressing digital video signals according to MPEG2. Then, the A_PCK and the V_PCK have the similar construction and there are some cases where the audio elementary stream contains a sub-stream ID according to the audio compression method.
Further, according to a timing rule for realizing synchronous playback using time stamp, or a rule for memory amount defined for sequentially reproducing coded data, the system target decoder (STD) as shown in FIG. 16 is modeled.
FIG. 16 is a block diagram illustrating an STD according to the MPEG2-PS (Program Stream). An access unit of an audio elementary stream is defined as being instantaneously transmitted from an input buffer to a decoder at a time corresponding to tpn to be decoded, and this tpn is coded into a bitstream as a PTS. On the other hand, for the video elementary stream, the I picture (Intra Picture) and the P picture (Predictive Picture) are decoded prior to B picture (Bidirectionally Predictive Picture), and therefore a decode time does not coincide with a display time. Therefore, data of the I picture and the P picture are defined as being instantaneously transmitted from an input buffer to a decoder at a time corresponding to td1 to be decoded, and the td1 is coded into a bitstream as a DTS, and these data are displayed at a time indicated as the PTS.
Many of the conventional techniques relating to the MPEG is directed to synchronously reproduce the respective stream data into which signals are multiplexed as MPEG streams with higher precision, and as a technique which has an object other than obtaining synchronization, for example, like a playback apparatus disclosed in the Japanese Published Patent Application No. Hei. 8-298645, which discloses a playback apparatus which plays sub-videos a predetermined time period later after it simultaneously plays main videos and audios, and completes the playback of the sub-videos and then simultaneously plays main videos and audios again.
However, while DVD enables various playback functions to be handled, how the contents can be reproduced just depends on the intention of the content producer, and even for the digital broadcasting, the broadcast station or program producer must add additional information to the content data so that the viewers can utilize various services.
For example, it is assumed that a state of a lecture meeting in which materials are projected onto a screen using a liquid crystal projector or an OHP for presentation is photographed and recorded using a video recorder. The viewer who plays the recorded video sometimes wants to view the video which has been played before, such as the materials of the presentation or the posture of the presenter giving the lecture, instead of the video which is played in synchronization with audio, while normally hearing audio of such as the presenter's lecture. Further, in a case where a parent other than a photographer views the video which is obtained by photographing and recording his child's music program held in a kindergarten or a nursery school, the parent sometimes wants to selectively view the video of a portion which is played with his child being centered while normally hearing the audio such as music. Further, also when viewing a live sport such as baseball or soccer on TV, the viewer may sometimes want to view the video of a highlight scene which has been broadcast before again while hearing audio live as it is. When viewing a live news program and the like, the same thing can be said. In either case, since the portions which the respective viewers want to view are various, the viewers sometimes want to carefully view the video other than the video photographed by a photographer or the video provided by the broadcast station with the audio being played as it is, the viewers feel frustrated.
While a novel playback function can be provided to viewers using the data reproduction method disclosed in the above-described patent document, the sub-video is played a predetermined time period later than the main video in this playback function, and the viewers cannot freely make operation for playing the video without feeling stressful.